Method for data communication

ABSTRACT

In order to enable high speed, high bandwidth data transfer between two ASIC devices, for example in a backplane, a wide parallel input data word is divided into a smaller number of words, and each smaller word is converted to serial form and then transmitted over a respective sub-link at a high clock rate relative to the system clock. At the receiving side, the clock is recovered from the serial words, and the serial words are converted back to parallel form. An alignment process is then carried out, firstly involving detecting the positions of the bits of the words and then storing the words in a buffer FIFO register. The words are clocked out of the FIFO register in synchronism under control of the system clock once it is detected that valid words are received in the FIFO registers.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.09/808,664 filed Mar. 15, 2001, which claims priority to Great BritainApplication No. 0006291.9 filed Mar. 15, 2000, each of which is herebyincorporated by reference in its entirety.

BACKGROUND

1. Field of the Invention

The present invention relates to a data communication link for highspeed, high bandwidth applications.

2. Description of the Related Art

In applications such as providing a data communication link between twoApplication Specific Integrated Circuits (ASICs) in a local backplane ofa computing system, very high data rates may be required, e.g., anaverage data rate of at least 4.8 Gigabits per second (Gbps). The datalink may be 64 bits wide.

Of the various possibilities for implementing such a link, it ispossible to provide an interface that transfers data from thetransmitting ASIC to the receiving ASIC as a single parallel word with asynchronising clock signal running at the system clock rate CK, say 78MHz. However, for a data word of 64 bits to achieve a data transfer rateof 4.8 Gbps this would require 65 device pins, which for manyapplications would be either impractical or too costly to provide in theASICs.

A synchronous interface could be used using a smaller number of pins, bymultiplexing a 64 bit wide data word N times onto W bits (=64/N) and byproviding a synchronising clock. However with a clock signal running at78 MHz, the bandwidth would be reduced to W*CK=BW/N, which would give anunacceptably slow data transfer rate.

In order to achieve a bandwidth of 4.8 Gbps, the transfer rate may bemultiplied N times. A synchronous interface which has a resultantTransfer Clock, N*CK, of less than 200 MHz may be practical. Above 200MHz, which would be necessary to achieve the desired transfer rate of4.8 Gbps, each data bit would be valid for a maximum of 5 ns, reducingfurther when rise-fall times of the interconnect signals andinput/output buffers are included. The task of achieving a robustdesign, ensuring that all W bits are aligned such that the synchronisingclock can always capture valid data bytes at the receiving ASIC, is farfrom trivial.

SUMMARY OF THE INVENTION

With a view to avoiding the above noted problems, the invention providesa method for transmitting data, comprising the steps of: (a) responsiveto a system clock, generating a transfer clock at a high rate relativeto the system clock; (b) dividing an input word into a plurality ofsmaller words; and (c) transmitting the plurality of smaller words overcorresponding serial sub-links in response to the transfer clock.

In an alternative embodiment, the invention further provides a methodfor receiving data, comprising the steps of: (a) converting receivedserial data words from a plurality of serial sub-links into parallelform; and (b) responsive to the received data, generating a low speedclock with a frequency nominally equal to a system clock.

The invention further provides a data transmitter having: a transferclock generator, responsive to the system clock, generating a transferclock at a high rate relative to the system clock; and a parallel toserial register, for dividing an input word into a plurality of smallerwords and transmitting them over corresponding serial sub-links inresponse to the transfer clock.

In an alternative embodiment, the invention still further provides areceiver having: a plurality of serial to parallel registers coupled tocorresponding serial sub-links, for converting received serial datawords from the sub-links into parallel form; and a clock generator,responsive to the received data, for generating a low speed clock with afrequency nominally equal to the system clock.

BRIEF DESCRIPTION OF THE DRAWINGS

A preferred embodiment of the invention will now be described withreference to the accompanying drawings wherein:

FIG. 1 is a schematic view of the transmit interface of a first ASIC ofthe data communication link of the invention;

FIG. 2 is a schematic block diagram of the receive interface of a secondASIC of the data communication link of the invention;

FIG. 3 is a more detailed diagram of the control mechanism for aligningreceived words in each sub-link of the link of FIGS. 1 and 2; and

FIG. 4 is a schematic block diagram of a Clock Data Recovery Module(CDRM) used in both the interfaces of FIGS. 1 and 2.

DETAILED DESCRIPTION

Referring to FIGS. 1 and 2 of the drawing, a Link Interface betweenfirst and second ASICs 2, 4 includes an interface 6 in ASIC 2. In oneembodiment:DW=W*NWhere;

DW=Bit width of wide data word

W=Bit width of sub-data word

N=An integer value, greater than 1

The data bandwidth across the link is given as;BW=DW*CKWhere;

BW=Bandwidth in Mega bits per second (Mbps)

CK=Transfer Clock in Mega Hertz (MHz)

For example, W=8, N=8, DW=64. The Transfer Clock, CK, is 78 MHz giving aBW of 4992 Mbps. However the invention is not limited to these specificvalues.

Interface 6 in ASIC 2 has a register 8 for breaking down the wide inputdata words, DW, into N (in this embodiment 8) smaller sub-words W (each8 bits long). Each sub-word W is treated independently, using a ClockData Recovery Module 10 (CDRM) macrocell. CDRM 10 has a multiplier 12for multiplying the clock CK, W (8) times and respective parallel toserial (PISO) converters 14 for operating on each of N, W bit words.Each serial word is transmitted over a respective sub-link 16.

Referring to FIG. 2, the receive ASIC 4 has an interface 20. The seriallinks 16 are coupled to another CDRM macrocell 22, in which a parallel Wbit word and clock is recovered for each of the N serial links.

FIG. 4 shows in more detail a CDRM 10, 22. The module 10, 22 has twoprimary functions. In transmit, it takes Low Speed Parallel Data (LDTX)on line 40 and creates High Speed Serial Data (HDTX) on line 42. Inreceive, it operates in reverse, taking High Speed Serial Data (HDRX) online 44 and creating Low Speed Parallel Data (LDRX) on line 46. Inaddition, the receive operation also recovers a Low Speed Clock (LDCK)on line 48 from the serial data, that is phase aligned with the LDRXdata. A Reference System Clock (REFCK) on line 49 is applied to a PhaseLocked Loop 50 which multiplies the clock rate by a factor of 8 toprovide a High Speed Clock (HSCK) on line 52. HSCK is applied to aparallel to serial register 54 and to a serial to parallel register 56.HSCK is also applied to a divide by 8 unit 58 and a chain of threetoggles 60. The outputs of toggles 60 are detected by an edge detectordevice 62 which provides an output to divider unit 58. The output ofdivider unit 58 comprises the Low Speed Clock (LDCK) on line 48. Theoperation of the circuit of FIG. 4 is as follows:

For Transmit, Low Speed Data (LDTX) on line 40 will be presented to theCDRM at the rate of the Reference Clock (REFCK). The Reference Clockwill be multiplied in frequency eight times by Phase Locked Loop (PLL)50 to create High Speed Clock (HSCK) on line 52. LDTX data on line 40will be loaded into a Parallel Serial Output (PISO) register 54 at theREFCK rate, and clocked out serially at the HSCK rate to form HDTX dataon line 42.

For Receive, the High Speed Clock (HSCK) will be divided by eight at 58to create a Low Speed Clock (LDCK) output. However, the phase of thisclock must be adjusted so that its associated Low Speed Data (LDRX) isstable at the time of the active edge of LDCK. This is done by edgedetection and phase adjustment circuit 60, 62 which monitors the HighSpeed Data (HDRX) on line 44. HDRX is also passed into a Serial ParallelOutput (SIPO) register 56 to create the Low Speed Received Parallel Data(LDRX) on line 46. The output from the SIPO 56 will be enabled on theopposite edge to the active edge of its associated clock LDCK.

The number of transmit and sub-links are replicated 8 times in thisexample. However, there will generally only be a single PLL per CDRMmacrocell.

On the receive side, the serial links are passed through CDRM macrocell22, and a W bit word and clock will be recovered for each of the Nserial links. The CDRM 22 has no knowledge of the boundary between one Wbit word and the next within the serial data stream and it is thereforethe first task of the Interface 20 to identify the correct bit alignmentwithin each sub-link. Having recovered the W bit words for eachsub-link, all N of the W bit words have to be aligned and synchronisedto recreate the original DW width word.

The bit alignment is achieved by the transmit side sending consecutiveinitialisation words constructed by ASIC 2. These initialisation words(of W bits) have the property that however many times the word isshifted right or left within another word that is 2W bits wide, there isa unique position that defines the bit alignment. For example consideran initialisation word, for W=8, of “10111000”. A register 24 that is 2Wwords wide holds the previously received and Previous & Current Word BitAlignment 10111000xxxxxxxx 0 x10111000xxxxxxx 1 xx10111000xxxxxx 2xxx10111000xxxxx 3 xxxx10111000xxxx 4 xxxxx10111000xxx 5xxxxxx10111000xx 6 xxxxxxx10111000x 7currently received words of W bits as shown in the above table. Theinitialisation word is sent at least twice followed by anothersynchronisation word (user defined) delimiter to indicate the start oftransmission of true data. The position of the word is located in theregister by means of a state machine (not shown) and this information isrelayed to subsequent stages.

During transmission, each ASIC transmitting/receiving interface willrespectively create/recreate a cyclic redundancy code (CRC) from thetrue data. The CRC words are inserted at a pre-determined interval,programmed to both transmit and receive sides. After this interval thetransmitted CRC should equal the recreated CRC. If not, then either bitalignment has been lost or a corruption has occurred during thetransmission of the data. This provides an Integrity Check individuallyon each of the serial links.

Thus, as shown in FIG. 2, subsequent to parallel conversion in CDRM 22,the parallel words are placed in a bit alignment register 24 in eachsub-link for detecting bit alignment. This is effected by a statemachine (not shown) locking onto the position of the initialisation wordwithin the register, and passing the bit aligned word to the next stage.In the next stage, an Integrity Check is performed on the CRC word at26.

The bit alignment and the Integrity Check are performed in each sub-linkusing the recovered clock generated for that serial link. There is noguarantee of any phase relationship between any of the N recovered clock(RCK[n])s, and each of the recovered clocks may be jittering (exceptthat the recovered clocks will be within one clock cycle of oneanother). However, the average frequency of all recovered clocks andthat of the Transfer Clock, CK, on the transmit side must be exactly thesame, since the reference clock to both the transmit and receive ASICswill be driven from the same crystal oscillator. A mechanism istherefore required to re-align the N recovered sub-words andresynchronise the wide data word back to the Transfer Clock, CK. This isdone by using a short First In First Out (FIFO) 28, 6 words long, at theend of each serial link.

The recovered sub-word plus a marker bit (W+1 bits) is written to theFIFO 28 by its associated recovered clock on line 48. The marker bitindicates whether that data word was Transmitted Synchronisation orIntegrity Check Word. The very first word to be written by each of thelinks, will be a synchronisation word (marker bit set) and the secondwill be the first sub-word of true data. The first write will occur at aslightly different time for each link, but by the time the second writeoccurs, all will have written at least once. The addressing of the FIFOsmay use Johnson coding, as more clearly seen in FIG. 3. An addressgenerator 32 provides a Johnson scheme of addressing to the write readaddress 34 of the respective FIFO 28.

The initial value of the address is 011 and the address scheme changesas indicated in FIG. 3. The most significant bit of the addresses of thesub-links are coupled by lines 36 to an OR gate 38. The output of the ORgate 38 is coupled by two metastability registers 70 to provide atrigger signal on line 72 to a state machine 74. State machine 74provides an output on line 76 to control the reading out of the FIFOregisters 28.

Thus, only a single address bit of FIFO's 28 changes per write and byensuring that the top address bit is set on the second write, thataddress bit can be logically OR'd with the equivalent bit from all Nlinks. This single bit signal, which goes high when the first word in asub-link is received, is resynchronised via the metastability registers70. By this time, since it is known all FIFO registers will be writtento within a clock cycle of one another, all FIFO's will contain words,and the state machine 74 triggers the Word Aligner to read from all NFIFO's in parallel at the Transfer Clock rate, CK. This read shouldtherefore occur when each of the FIFOs contain approximately four words.As the average frequency of the read and write clocks to the FIFO is thesame each FIFO should always contain approximately four words. A FIFOthat is at least six deep will isolate against jitter on the recoveredclocks.

The very first FIFO read will all be synchronised sub-words but thesecond will be the recovery of the first true wide data word. The outputof the FIFOs are applied to a Word Aligner register 78 whichreconstitutes the original data word 80 (FIG. 2). Word Alignment ischecked at the same programmed interval used by the bit alignment,because at this time, and only at this time, all of the marker bits ineach of the N FIFO's will be set.

The scheme outlined provides a robust high speed, high bandwidth locallink by using a number of serial asynchronous links in parallel.

Thus, it will now be understood that there has been disclosed a newmethod and apparatus for providing a data communication link. While theinvention has been particularly illustrated and described with referenceto preferred embodiments thereof, it will be understood by those skilledin the art that various changes in form, details, and applications maybe made therein. It is accordingly intended that the appended claimsshall cover all such changes in form, details and applications which donot depart from the true spirit and scope of the invention.

1. A method for transmitting data over a communication link between afirst integrated circuit having a system clock and a second integratedcircuit, comprising the steps of: a. responsive to the system clock,generating a transfer clock at a high rate relative to the system clock;b. sending one or more bit alignment code words to initialize thecommunication link; c. dividing an input word into a plurality ofsmaller words; and d. transmitting the plurality of smaller words overcorresponding serial sub-links in response to the transfer clock.
 2. Themethod of claim 1, further comprising the step of: e. transmitting a CRCcode word at predetermined intervals.
 3. A method for receiving datatransmitted over a communication link having a plurality of serialsub-links at an integrated circuit having a system clock, comprising thesteps of: a. receiving a plurality of serial data words over theplurality of serial sub-links; b. responsive to the plurality of serialdata words, generating a low speed clock with a frequency nominallyequal to the system clock; c. storing the received serial data words foreach sub-link in a plurality of buffer memories correspondingrespectively to each sub-link; and d. reading the buffer memories insynchronism under control of the system clock in order to reconstituteeach data word in parallel form.
 4. The method according to claim 3,wherein the buffer memories are FIFO registers.
 5. The method accordingto claim 4, wherein the step c. of storing received data words comprisesthe step of: e. addressing the buffer memories by changing only one bitof the address for each incremental address.
 6. The method according toclaim 5, wherein the step d. of reading the buffer memories comprisesthe steps of: f. comparing a predetermined bit of the address of eachFIFO; and g. generating a trigger signal to actuate a state machine tocause reading of the FIFO registers.
 7. A method for receiving datatransmitted over a communication link having a plurality of serialsub-links at an integrated circuit having a system clock, comprising thesteps of: a. receiving a plurality of serial data words over theplurality of serial sub-links; b. storing the received serial data wordsfor each sub-link in a plurality of serial-to-parallel registerscorresponding to the serial sub-links; c. responsive to the plurality ofserial data words, generating a low speed clock with a frequencynominally equal to the system clock; d. detecting one or more edges ofthe received serial data words; e. based on the one or more detectededges, aligning the low speed clock with the plurality of serial datawords; and f. applying the low speed clock to the plurality ofserial-to-parallel registers for clocking out parallel words from theplurality of serial-to-parallel registers.
 8. The method according toclaim 7, further comprising the step of: g. storing the received bitalignment words in a bit alignment register in order to locate theposition of the bits in the plurality of serial-to-parallel registers.9. The method according to claim 8, further comprising the steps of: h.generating a CRC code word in response to the received data; and i.checking a received CRC code word against the generated CRC code word.10. A method for communicating data over a communication link between afirst integrated circuit having a first system clock and a secondintegrated circuit having a second system clock, comprising the stepsof: a. responsive to the first system clock, generating a transfer clockat a high rate relative to the first system clock; b. sending one ormore bit alignment code words from the first integrated circuit to thesecond integrated circuit to initialize the communication link; c.dividing an input word into a plurality of smaller words; d.transmitting the plurality of smaller words from the first integratedcircuit to the second integrated circuit over a corresponding pluralityof serial sub-links in response to the transfer clock; e. receiving, atthe second integrated circuit, the plurality of smaller words over theplurality of serial sub-links; f. responsive to the plurality of serialdata words, generating a low speed clock with a frequency nominallyequal to the second system clock; g. storing the received serial datawords for each sub-link in a plurality of buffer memories correspondingrespectively to each sub-link; and h. reading the buffer memories insynchronism under control of the second system clock in order toreconstitute each data word in parallel form.